Method and system for automatic identification and orientation of medical images

ABSTRACT

A method and system for computer-aided detection of abnormal lesions in digital mammograms is described, wherein digital films are processed using an automated and computerized method of detecting the order and orientation of a set of films. In one embodiment, anatomic features are used to detect the order, orientation and identification of a film series. In another embodiment of the invention, a technologist feeds films into the system in any order and orientation. After processing, the system provides an output on a display device to a radiologist that is in an order and orientation preferred by the radiologist. In yet another embodiment of the invention, films from one case are distinguished from films of another case. In this manner and through the use of a bulk loader, a large number of films can be stacked together and fed into the system at one time.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of application Ser. No. 09/721,347,filed Nov. 22, 2000 now U.S. Pat. No. 7,146,031, a copy of which isincorporated by reference.

FIELD OF THE INVENTION

The present invention relates to the field of Computer-Aided Detection(CAD) of abnormalities found in medical images and to the field ofPicture Archiving and Communication Systems (PACS). In particular, theinvention relates to a method and system for determining the orientationof digitized medical images in order to enable consistent andunderstandable communication between a computer-aided detection systemand an operator. The invention further relates to a method and systemfor uniquely identifying and distinguishing images from a particularcase.

BACKGROUND OF THE INVENTION

Systems for computer-aided detection (“CAD”) assist radiologists in thedetection and classification of abnormal lesions in medical images. Thepurpose of such devices, as described in U.S. Pat. No. 5,815,591 toRoehrig, et. al., entitled “Method and Apparatus for Fast Detection ofSpiculated Lesions in Digital Mammograms,” the disclosure of which ishereby incorporated by reference in the present application, is todirect the attention of a radiologist to suspicious areas of the medicalimage that may reflect a threatening condition. While not a replacementfor the experienced radiologist, CAD systems are designed to increaseefficiency and reduce error, as a typical radiologist may be required toexamine hundreds of medical images per day, which can lead to thepossibility of a missed diagnosis due to human error.

Various systems and methods are currently known for computerizeddetection of abnormalities in radiographic images, such as thosedisclosed by Giger et al. in RadioGraphics, May 1993, pp. 647-656; Gigeret al. in Proceedings of SPIE, Vol. 1445 (1991), pp. 101-103; U.S. Pat.No. 4,907,156 to Doi et al.; U.S. Pat. No. 5,133,020 to Giger et al.;U.S. Pat. No. 5,343,390 to Doi et al.; U.S. Pat. No. 5,491,627 to Zhanget al. These references are incorporated herein by reference as thoughfully set forth herein. These systems are generally referred to asComputer-Aided Diagnosis systems, Computer-Aided Detection systems, orsimply, CAD systems. Such systems are believed to be particularly usefulto radiologists and other medical specialists in diagnostic processesand specifically in radiologic screening procedures.

In a radiologic screening procedure, such as screening mammography, trueabnormalities such as cancers are believed to occur at a typical rate ofabout one case per two hundred patient examinations. It is believed aCAD system, serving as an electronic reminder or second reader, canassist radiologists in obtaining higher detection rates, or highersensitivity for abnormalities. Additionally, such CAD systems can assistradiologists in reducing the misdiagnosis rate, or lowering the falsenegative rate. Thus, it is believed that the use of such CAD systemswill continue to increase.

Since such CAD systems typically operate on medical images inhigh-resolution digital format, film-based medical images ordinarilymust be scanned by a high resolution scanner to convert the image datainto digital form. With current CAD systems, however, we have found thatsystems for loading and feeding film-based medical images to the scannerare inadequate in that they tend to require too much time and effortfrom the user. Additionally, we have found that currently availablesystems do not allow the user to simply and conveniently enter caseinformation, monitor the status of cases, and abort or adjust theprocessing of films being processed. For example, with currentlyavailable systems, errors in film orientation may not be detected untilthe radiologist views the analyzed image.

Using a light box, radiologists typically look at X-ray films such asmammograms or chest images, in a very well-defined orientation andorder. For example, a mammogram typically contains four (4) films, theCranial Caudal (“CC”) and Medio-Lateral Oblique (“MLO”) views of each ofthe left and right breasts. In most clinics the four films are arrangedon a light box in a particular order and orientation, however in otherclinics the films may be in a different order and orientation. Theimportant point however, is that a radiologist in a given clinic is veryaccustomed to the same, unchanging order and orientation. When acomputer is used to digitize, analyze, and/or display the images with orwithout some annotation, it is important that the computer display thesame order and orientation of the film series that a doctor isaccustomed to using to view the films.

One way of insuring correct order and orientation is to require that theimages be input into the computer in a predefined order and orientation.This in fact is the present method used by the Computer Aided Detection(CAD) device ImageChecker, as described in U.S. Pat. No. 5,729,620 toWang entitled “Computer-Aided Diagnosis System and Method,” thedisclosure of which is hereby incorporated by reference in the presentapplication. Experience has shown, however, that this requirement placessome burden on the technologist operating the system, adding to the timerequired to do the work, and occasionally being a source of error.

For the technologist to align all films in each case in a prescribedmanner would be onerous, time consuming, and prone to error.Statistically, with any order and any orientation equally likely, theprobability that a random order of the four films of a standardmammogram is correct can be expressed asp=1/(8⁴×4!)=1/(4096×24)=1/98,304=1.02×10⁻⁵ where four (4) films can bein 8 orientations and placed in four (4) different positions in anorder. Thus, it would be advantageous to input a set of films in anyorder or orientation and have them processed and displayed in apreferred order and orientation.

SUMMARY OF THE INVENTION

Therefore, an object of this invention is to provide an automated,computerized method of detecting the order and orientation of a filmseries using predetermined film identifying marks and rearranging thatorder and orientation into any desired order and/or orientation. Inanother embodiment of the invention, anatomic features are used todetect the order, orientation and identification of a film series.

The present invention provides advantages in terms of time savings aswell as reduction of errors on the part of a technologist and/or user ofthe system. It allows a technologist to feed films into the system inany order and orientation while providing an output to a radiologistthat is in an order and orientation preferred by the radiologist. It isanother object of the present invention to provide a method todistinguish the films belonging to one case (i.e., one patient) fromthose of another case. These advantages are to be further appreciatedwhen a large number of cases are fed into a digitizer at one time with abulk loading input feeder (i.e., bulk loader). With a bulk loader, alarge number of films, typically on the order of 100, can be stackedtogether and fed into the system at one time in a manner similar to highcapacity paper copying machines. The stack of 100 films would containmany cases (e.g. 25) from different patients, acquired over a reasonablylong period of time.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, reference is be made to thefollowing detailed description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram of an embodiment of a computer aided detectionsystem of the present invention;

FIG. 2 is block diagram of functional units of an embodiment of thepresent invention;

FIG. 3 is a diagram of a an embodiment of a computer aided detectionsystem of the present invention;

FIG. 4 is diagram of a processing unit of the present invention;

FIG. 5 is a diagram of an embodiment of a computer aided detectionsystem of the present invention;

FIG. 6 is flowchart of a method of the present invention;

FIG. 7 is a diagram of mammographic films as used in the presentinvention;

FIG. 8 is a diagram of lead markers as used in the present invention;

FIG. 9 is a diagram of mammographic films with a bar code label as usedin the present invention;

FIG. 10 is a diagram of mammographic films with a cover sheet as used inthe present invention;

FIG. 11 is a flowchart of a method for determining a case according toan embodiment of the invention;

FIG. 12 is a diagram of mammographic films with a unique identifier asused in the present invention;

FIG. 13 is a flowchart of a method for determining a chest wall side ofa mammographic film according to an embodiment of the invention;

FIGS. 14-16 are flowcharts of a method for identifying charactersimprinted on a mammographic film according to an embodiment of theinvention; and

FIG. 17 is a flowchart of a method for displaying digitized images in apreferred or predetermined order and/or orientation.

DETAILED DESCRIPTION OF THE INVENTION

It has been found that the currently available CAD systems are lackingin their ability to facilitate the loading and feeding of films forprocessing. Furthermore, it has been found that such systems furtherlack in their ability to display digitized images to a doctor orradiologist. In particular, it has been found that providing a filmfeeding mechanism that holds multiple films and automatically feeds thefilms to the scanner greatly reduces the time and labor required to loadand input films into the system. It has been found that a stack filmfeeder that holds a relatively large number of films makes the inputtingof films much more efficient. Furthermore, it has been found that asystem that allows for displaying digitized films in various ordersand/or orientations provides for a more user-friendly interface.

FIG. 1 shows an outside view of a computer aided detection (CAD) system100, such as the Image Checker M1000 from R2 Technology, Inc., forassisting in the identification of suspicious lesions in mammograms. CADsystem 100 comprises a CAD processing unit 102, an input 106, and aviewing station 104. In general, CAD processing unit 102 scans aconventionally developed x-ray mammogram 101 that is provided at input106 into a digital mammogram image, processes the image, and outputs ahighlighted digital mammogram 101D for viewing at viewing station 104.

Each digital mammogram may be, for example, a 4000×5000 array of 12-bitgray scale pixel values. Such a digital mammogram would generallycorrespond to a typical 18 cm×24 cm x-ray mammogram which has beendigitized at a 50 micron spatial resolution. Because a full resolutionimage such as the 4000×5000 image described above is not alwaysnecessary for the effectiveness of the preferred embodiments, the imagemay be locally averaged, using steps known in the art, down to a smallersize corresponding, for example, to a 200 micron spatial resolution.

FIG. 2 shows a block diagram of CAD processing unit 102. CAD processingunit 102 comprises a digitizer 203, such as a laser scanner with 50micron resolution, for digitizing the conventionally developed x-raymammogram 101. CAD processing unit 102 generally includes elementsnecessary for performing image processing including parallel processingsteps. In particular, CAD processing unit 102 includes elements such asa central control unit 205, a memory 208, a parallel processing unit210, and an I/O unit 212. It is to be appreciated that the parallelprocessing unit 210 shown in FIG. 2 may be replaced by a singleprocessor without departing from the scope of the preferred embodiments.It is to be appreciated that in addition to the suspicious lesiondetection algorithms disclosed herein, processing unit 102 is capable ofperforming a multiplicity of other image processing algorithms eitherserially or in parallel therewith.

Viewing station 104, as shown in FIG. 1, is for conveniently viewingboth the conventionally developed x-ray mammogram 101 and the digitalmammogram 101D that is formed by the CAD processing unit 102. Theconventionally developed mammogram is viewed on a backlighting station120. The digital mammogram is displayed on a display device 118 that maybe, for example, a CRT screen. The display device 118 typically shows ahighlighted digital mammogram 101D corresponding to the x-ray mammogram101, the highlighted digital mammogram having information directing theattention of the radiologist to suspicious areas as determined by imageprocessing steps performed by the CAD processing unit 102. In onepreferred embodiment, the highlighted digital mammogram will have anasterisk (*) superimposed on those locations corresponding to suspiciouslesions, and a triangle (Δ) superimposed on calcification clusters.Further information regarding CAD system 100 may be found in U.S. Pat.No. 5,815,591, supra.

It is to be appreciated that in addition to being able to display asingle view of one breast, CAD system 100 may be used in accordance withthe preferred embodiments to simultaneously display information relatedto multiple views of the same breast, similar views of both breasts,and/or views of a single breast taken at different points in time. Thus,the attention of the radiologist may be drawn to specific areas of afirst mammogram image by CAD system 100, which can then be compared tocorresponding areas of other views of the same breast, views of theother breast, or previous views of the same breast for making anappropriate determination.

According to another embodiment of the invention, a system for thedetection of breast carcinoma is illustrated in FIG. 3. As shown,detection system 300 comprises a processing unit 302, a motorized viewer304, and a wet read viewer 306. According to this embodiment,mammography x-ray films are loaded, scanned and analyzed by a processingunit 302. It is noted that the functions performed within processingunit 302 are similar to those performed within processing unit 102. Oneof skill in the art understands that each unit can be modified toperform identical functions. After the x-ray films are analyzed byprocessing unit 302, data representing regions of interest identified bythe processing unit are transmitted for display to one or more viewers.In the example shown in FIG. 3, the data is be sent to motorized viewer304, wet read viewer 306, or both.

Note that although these preferred embodiments are described withrespect to detection systems that process and analyze mammography x-rayfilms, the present invention is readily adaptable to many other types ofCAD systems. Additionally, the present invention is applicable to CADsystems which analyze other kinds of images besides x-ray films. Theinvention is applicable to CAD systems which process any type offilm-based medical images. For example, ultrasound imaging, nuclearmagnetic resonance imaging, computer tomography imaging, andscintillation camera imaging all produce images which are film-based.Additionally, film-based medical images are carried on a wide variety offilm-like materials such as vellum, or any other transparent ortranslucent media.

Referring now to FIG. 4, the components in processing unit 302 and wetread viewer 306 will now be described in greater detail according to apreferred embodiment of the invention. As shown, processing unit 302houses a user interface 402, scanning unit 404, and computer 406 Notethat although the scanning unit and processing unit are housed withinprocessing unit 302, they can also be provided separately or in othercombinations. In general, processing unit 406 comprises a computer-basedsystem for the detection of anatomical abnormalities. However,processing unit 406 is generally the processor of almost any CAD system.For example, the present invention is applicable to and could be adaptedto facilitate the input of film-based images into any of the CAD systemsmentioned and incorporated by reference above.

As shown in FIG. 4, user interface 402 contains a display panel 408which is a touch sensitive flat panel display and wet read viewer 306 isanother touch sensitive flat panel display where both display panel 408and wet read viewer 306 are placed in close proximity so that a user isable to access both with ease. An alternative embodiment of theinvention implements a touch sensitive flat panel display to function asboth the display panel 408 and the wet read viewer 306.

User interface 402 comprises an X-ray film feeding mechanism 410 whichhandles and feeds the films in serial fashion to scanning unit 404. Filmfeeding mechanism 410 comprises a stack film feeder 412 that is capableof holding a large number of films. According to a preferred embodimentof the invention, film feeding mechanism 410 is designed so as toaccommodate cases of films where each case of films is made up of anumber of films obtained from a single patient. For example, in theUnited States, it is common for a case to be composed of fourmammography x-ray films. Each breast is usually imaged twice: the firstimage being a top view ordinarily called the Cranial Caudal view (“CC”),and the second image being a lateral view ordinarily called theMedio-Lateral Oblique view (“MLO”). The invention can accommodate casescomposed of a number of films up to the maximum capacity of the stackfilm feeder 412.

According to a preferred embodiment, film feeding mechanism 410comprises a stack film feeder 412 capable of feeding relatively largenumbers of films. Stack film feeder 412 individually feeds films toscanning unit 404 as required during processing on either afirst-in-first-out (“FIFO”) basis or last-in-first-out (“LIFO”) basis.Suitable stack film feeders 412 are currently or will soon becommercially available from vendors such as Konica, Canon and AbeSekkei. Sometimes such feeders are called bulk loaders or stack loaders.Note that in embodiments where each film has its own bar code label, abar code reader is provided in stack film feeder 412. Preferably,graphical or electromechanical user interface implemented motorizedviewer 304 or wet read viewer 306 controls the organization of imagesprocessed en masse.

Additionally, film feeding mechanism 410 is designed so as toaccommodate the size of films ordinarily used in a particularapplication. For example, in a preferred embodiment, stack film feeder412 is designed to hold either 18 cm×24 cm, or 24 cm×30 cm films. Thisis accomplished by providing a film feeding mechanism with a throat thatis 24 cm wide to accommodate both film sizes, in which case the 18 cm×24cm films are rotated in software using standard digital image rotationtechniques. Furthermore, a preferred embodiment of the inventionutilizes a cover sheet with a tab 1010 (see FIG. 10) that protrudesbeyond the other films. Thus it is necessary that the film feedingmechanism be designed so as to accommodate the protruding tab.

Transport mechanism 424 transports the films individually from filmfeeder 412 to scanning unit 404. After the scanning unit 404 hascompleted scanning a film, the film is ejected to an output film holder426, and transport mechanism 424 feeds the next film to scanning unit404. According to a preferred embodiment, film feeding mechanism 410 andtransport mechanism 424 comprises a commonly available film feeding unitsuch as those commercially available from Konica, Canon and Abe Sekkei.

Scanning unit 404 generates from each x-ray film a two-dimensionalmammographic image. Preferably, scanning unit 404 is a laser filmdigitizer and has a dynamic range and spatial resolution comparable tothose of the original mammographic film which typically has a dynamicrange of 10,000:1 and spatial resolution of approximately 50 microns perpixel.

Although film feeding mechanism 410 and scanning unit 404 are describedherein according to certain preferred embodiments, in general manyalternative types of feeding mechanisms and scanners can be used,depending on the particular application. For example, suitable scannersare commercially available from a number of vendors including Konica,Canon and Abe Sekkei.

Additionally, certain medical images are already in digital format, suchas images that were acquired with a digital medical imaging system, orthat are stored in a digital image storage system. According to theinvention, an example of a computer-aided detection system whichreceives images already in digital format is shown in FIG. 5. Althoughprocessing unit 302 is shown connected to both digital image storagesystem 502 and digital medical imaging system 504, in general only onesource of digital image data is needed. In another embodiment of theinvention, user enhancements in processing unit 302 and viewer 306 canbe removed especially where the images in digital format are not proneto error. A wide variety of digital medical imaging systems currentlyexist. Some examples are: computer tomography systems, digitalultrasound imaging systems, scintillation camera systems, digitalstimulated emission phosphor plate radiography systems, nuclear magneticimaging systems, and digital mammographic systems. An example of adigital image storage system is disclosed in U.S. Pat. No. 5,416,602 toInga et al., entitled “Medical Image System With Progressive Resolution”incorporated herein by reference. In the case where medical images arealready in digital format, the feeding and scanning functions of thesystem are not needed. In such cases, the operator monitors the digitaldata being received by the system using the display panel, and is ableto re-orient or change the order of images electronically, as will bedescribed in greater detail below.

FIG. 6 shows the general steps performed by CAD processing unit 102 onthe x-ray mammogram. Reference will be made to processing unit 102,however, one of skill in the art understands that the discussion isapplicable to processing unit 302 as well. At step 602, multiple relatedx-ray mammograms are scanned in and digitized into digital mammograms.To create the x-rays mammograms, the breast is positioned and compressedbetween two plates before the X-ray is actually taken. Two views,corresponding to two approximately orthogonal orientations of thecompression planes are taken, typically called Cranial Caudal (CC) viewand Medio-Lateral Oblique (MLO) view. The resulting four films aredeveloped, and digitized by the CAD system 100. The digitized image isthen sent to a computer for processing at step 604.

The digital mammograms are processed at step 604 by an overallsuspicious lesion detection algorithm in accordance with the preferredembodiments. The overall lesion detection algorithm performed at step604 generates a list of locations in at least one of the digitalmammogram images which correspond to suspicious lesions, i.e. possiblycancerous lesions. The algorithm operates independently on each image,without regard for what lesions are found or not found in the otherimages. Following step 604, the digital mammogram images and list ofsuspicious locations is sent for display to the viewing station 104 atstep 606.

At step 606, the system of the present invention displays a digitizedimage of the breast mammogram with indicators overlaid on the imageindicating regions of interest. At step 606, the doctor also hasavailable the original conventionally developed mammographic films towhich he can refer. After thoroughly examining the films and thedigitized images, the doctor makes a recommendation.

As an alternative to the manual x-ray development following by thedigitization performed at step 602, the x-ray detector, which at presentis usually a film screen cassette, can be replaced by a direct digitaldetector such as the General Electric Full Field Digital MammographySystem, and the resulting digital image fed directly to the CADprocessing unit 102.

While in one embodiment the multiple related x-ray mammograms correspondto two views of the same breast, e.g., the Cranial Caudal (CC) and theMedio-Lateral Oblique (MLO) view, in another embodiment the multiplerelated x-ray mammograms correspond to similar views of both breasts,e.g., the MLO view of the left and right breast of a single subject. Inyet another embodiment, there are three related x-ray mammograms whereinthe first two are different views of the same breast, e.g., the CC andthe MLO views, and the third is a view such as a the MLO view of theother breast. In still another embodiment, there is a fourth, historicalx-ray mammogram corresponding to the first x-ray mammogram except thatit has been taken months or years earlier in time. It is to beappreciated that there are further combinations of the above x-raymammograms to be input into the CAD processing unit 102 that are withinthe scope of the preferred embodiments. It is to be further appreciatedthat the historical x-ray mammogram that was taken months or yearsearlier in time can be stored in digitized form in CAD computer memory208, or other digital storage medium, without departing from the scopeof the preferred embodiments. Alternatively, to save storage space, onlya historical set of feature vectors derived from prior digitalmammograms can be stored in the CAD memory 208. Indeed, the CAD memory208 or other storage medium can contain an entire historical archive ofprior digital mammograms taken of the same breast or feature vectorsderived therefrom, and CAD processing unit 102 can be modified tocompare any of these historical digital mammograms or feature vectorsderived therefrom to the current digital mammogram.

The films in a film case are ordinarily arranged in a predeterminedorder. For example, in the United States, the four Cranial Caudal andMedial-Lateral Oblique x-ray views are ordinarily arranged in the orderRight Cranial Caudal (RCC) film 702, Left Cranial Caudal (LCC) film 704,Right Medial-Lateral Oblique (RMLO) film 706, and Left Medial-LateralOblique (LMLO) film 708 as shown in FIG. 7. Other orders, however, maybe used. In other circumstances, a case may contain 2 films (e.g., wherea patient who has only 1 breast), or even just 1 film. For example, inEurope, it is common for a case to include only one image per breast, inparticular the Medio-Lateral Oblique view. The actual number of filmsvaries and cannot be predicted a priori. An embodiment of the presentinvention therefore determines when one case begins and ends. In anotherembodiment, the present invention further determines how many films areassociated with a particular case.

In accordance with the invention, a variety of methods are available forcontrolling the order and orientation of films. Technologists typicallyplace lead (Pb) markers on x-ray film cassettes during an x-rayexposure. The lead markers then provide a permanent image on the x-rayfilm when it is developed. Technologists taking mammographic imagestypically use lead markers 802, 804, 806, 808 as shown in FIG. 8. RCCmarker 802 produces RCC lead marker image 703 shown in FIG. 7.Similarly, LCC marker 804, RMLO marker 806, and LMLO marker 808 of FIG.8 produce LCC lead marker image 705, RMLO lead marker image 707 and LMLOlead marker image 709 of FIG. 7. While the angular orientation of thelead markers on the films tends to be placed such that the markers areread from left to right, the orientation is, in some instances, random.The letters can also be mirror inverted if placed upside down. Theplacement of the lead markers, however, is specified by the AmericanCollege of Radiology (ACR) standards and published in “MammographyQuality Control” by the American College of Radiology Committee onQuality Assurance in Mammography, 1992 and subsequent years, the entirecontents of which are incorporated by reference. In particular, the ACRstandards specify that the lead markers are to be placed “near theaspect of the breast closest to the axilla”, or in practice, in the tophalf of the film as shown in FIG. 7 for lead marker images 703, 705, 707and 709. In accordance with the invention, orientation and identityinformation is obtained by detecting the presence of lead marker images703, 705, 707 and 709.

In an embodiment of the invention shown in FIG. 9, a bar code label 902is placed on the top left side of RCC film 702. Other bar code labels(not shown) are placed on the other films in a case. Each bar code labelcontains encoded or other information that is read by a machine or atechnologist. While the use of bar code labels will be described, one ofskill in the art will appreciate that other types of labels can be usedincluding magnetic readable labels. Furthermore, as other types oflabeling becomes available, they can also be used. In accordance withthe invention, orientation and identity information is obtained bydetecting the presence and/or contents of bar code labels 902.

When placing bar code labels on the films it is important that theirplacement does not cover or touch any necessary information. Thus it isimportant that the bar code labels not cover the imaged breast, leadmarker images or any other information on the films. Furthermore, theperformance of the present invention is enhanced if the bar code labelsare not too close to any edge of the film. While keeping with ACRstandards, it is suggested that bar code labels be placed in the topleft side of RCC film 702 and RMLO film 706 and the top right side ofLCC film 704 and LMLO film 708. One of skill in the art appreciates thatthe position of the labels can be changed as standards and other factorschange.

Further orientation and identity information is obtained by detectingthe presence of lead marker images 703, 705, 707 and 709 in conjunctionwith detection of the bar code labels. Algorithms that have been foundeffective in detecting the lead markers and bar code labels aredescribed further below.

An embodiment of the invention provides a method for separating one case(i.e., films associated to one patient) from another case in anautomatic, computerized way. The need to separate cases arises whenusing stack film feeder 412 (see FIG. 4). Stack film feeder 412typically can hold many cases, such as 100 films corresponding to 25cases, and sequentially feed them into a scanner which digitizes theimages and sends the digital images to a computer for furtherprocessing. For a technologist to align all films in each case in aprescribed manner is onerous, time consuming, and prone to error usingmethods in the prior art. Advantageously, the present invention providesa savings in time needed to feed films into a CAD system and furtherreduces errors in the feeding of films.

In particular, a method is provided for separating and grouping filmsassociated with a particular patient. According to an embodiment of theinvention as shown in FIG. 10, a different type of film is used as acover or separator sheet 1002 which is inserted before each case tofacilitate organization of the cases by the technologist. Further, asecond type of cover sheet (not shown) is provided for missing viewssuch as in the case of a missing breast due to a mastectomy or because aview was not imaged. While the requirements for the number of films in acase varies, stack film feeder 412 of FIG. 4 must accommodate theparticular requirements of the implementation.

As shown in FIG. 10, a case identification label 1012 that associatesthe films in the case with a specific patient is placed on the lowerright hand side of cover sheet 1002. In another embodiment of theinvention, cover sheet 1002 has tab 1010 on which case identificationlabel 1012 is placed. Other locations are also appropriate withoutdeviating from the present invention. As shown in FIG. 10, caseidentification label 1012 is a bar code label that can be read by a barcode reader. Alternatively, textual or encoded information is placed onlabel 1012. A patient identification label (not shown) can also beplaced on the film to explicitly identify the patient or provide textualinformation about the patient.

Advantageously, only the cover sheet 1002 has case identification label1012 affixed on it; and each case, whatever the number of films itcontains, has a cover sheet placed on top by the technologist beforeinserting it into the stack film feeder 412. The bar code is scannedeither by the film scanner or by a separate bar code scanning devicebefore the patient films are scanned and digitized. The informationrepresented by the bar code label is then associated with a particularset of films between the cover sheet to which the label is affixed andthe next cover sheet. Whenever the system of the present inventiondetects the presence of cover sheet 1002 or the presence of caseidentification label 1012, the system assumes that the films that followimmediately thereafter constitute a case that is different from thosethat precede the separator sheet. Thus, cover sheet 1002 or caseidentification label 1012 serves as a first-film indicator.

In other embodiments of the invention, case identification label 1012 isplaced on the first film of a case or on each film of the case. In FIG.4, a bar code reader is mounted in film feeding mechanism 410 to readthe cases as they are being fed in by the stack film feeder 412.Alternatively, scanning unit 404 could be adapted to read the bar codelabel from each film.

Other forms of indicating the first film of a case are possible withoutdeviating from the teachings of the invention. For example, as shown inFIG. 9, lead marker image 703, 705, 707 and 709 can be read using anoptical character recognition (OCR) algorithm. An OCR algorithm will bedescribed below. Where the films are known to be stored in a specifiedorder, the method of the present invention searches for a firstoccurrence of the lead marker image that indicated the first film.

Where the order of the films is not known, a first-film indicator can bedetermined by loading a number of films and identifying the associatedlead marker images until a complete set of films is loaded such as a setof RCC, LCC, RMLO and LMLO films. The first film of this set is thenidentified as the first film of the previously loaded case. It is notedthat although embodiments are described using a first-film indicator,the teachings can be extended to use other indicators such as a secondfilm indicator, third film indicator, or last film indicator.

In another embodiment of the invention as shown in FIG. 9, caseidentification label 902 is used as a first-film indicator. In yetanother embodiment of the invention shown in FIG. 12, a unique mark 1202is placed on a first film of a case. Unique mark 1202 can be a speciallyshaped hole such as can be made with a one-hole paper punch. Unique mark1202 is then used as a first-film indicator. Where a unique mark 1202 isused, the method of the present invention implements a technique such aspattern matching which will be further described below. Furthermodifications include creating a unique identification number andassociating it with a bar code which is then saved with the films. Thebar code can then be used for retrieval of information at a later time.

The modifications just described provide the further advantage that morecases can be loaded at one time since no space is occupied in thescanner feeder by cover sheets 1002. However, where a stack containingno cover sheets is used, a technologist loading and unloading the systemof the present invention needs to take particular care to physicallyseparate individual cases in the stack. Thus errors can be reduced byproviding a separator sheet 1002 of a conspicuous color or a cover sheet1002 with a tab 1010.

Referring to FIG. 11, there is shown a flowchart for an algorithm of anembodiment of the present invention. At step 1102 a film is loaded intothe system of the present invention. The system then searches for afirst-film indicator on the loaded film at step 1104. If a first-filmindicator is not found, the method again loads another film as shown byloop 1106. If a first-film indicator is found, the film is associatedwith a patient at step 1108. Another film is loaded at step 1110.Another search is made for a first-film indicator at step 1112. If nofirst-film indicator is found, the film that was just loaded isassociated with the same patient as shown by loop 1114. If a first-filmindicator is found, the film that was just loaded is associated withanother patient at step 1116. Another film is then loaded as shown byloop 1118. One of skill in the art understands that loop 1118 can berepeated many times to complete the loading of a large stack of films.One of skill in the art further understands that modification of themethod as shown in FIG. 11 is possible without deviating from theteachings of the invention.

After the case is read through the scanner, the case films are put intoa patient envelope along with the separator sheet containing the barcode. The information associated with a particular case is then directedto viewing station 104 (or motorized viewer 304) associated with apatient identification number. Subsequently, a doctor can review thefilms on viewing station 104. Furthermore, patient films can be hung onan integrated or nearby light box. Upon hanging of the films, automatedretrieval of information is accomplished by scanning the bar code on theseparator sheet, for example. The display unit then associates thatnumber with the corresponding space or “slot” on viewing station 104.The viewing station can then display the correct digital images when thedoctor requests information on the films from an identified slot.

A further object of the present invention is to determine theorientation of a film. Useful information about orientation can beobtained from anatomic or geometric features. For example, mammographicimages are rectangular in shape (typically either 18×24 cm or 24×30 cm).Furthermore, so as to maximize the imaging area of a radiographic image,the chest wall of a patient is usually along one of the longer edges ofa mammographic film. This information can then be used in an embodimentof the invention to determine the left or right orientation of a film.

As shown in the flowchart of FIG. 13, a digitized radiographic image issplit length-wise at step 1302 into two halves where each half is either9×24 or 12×30 depending on the film used. At step 1304, substantiallyall artifacts are removed. Artifacts include unexposed regions due tomasking and unexposed regions along the image edges. At step 1306, thepixel intensities in the two halves are summed. One side of the imagewill have a higher average pixel intensity than the other due to theattenuation of the x-rays by the breast. The pixel intensities are thencompared at step 1308. The side with the larger average pixel intensity(greater than some threshold value) is determined at step 1310 to be thechest wall side. In another embodiment, the side with a pixel intensitygreater than a predetermined threshold is associated with the chest wallside. Digitization of the images with a pixel size of 200 microns hasbeen found to be adequate for the purposes described here.

Those skilled in the art understand that other anatomical features arecommon in mammographic images. More generally, those of skill in the artunderstand that other types of radiographic images such as chest x-rayscontain common anatomical features which can be used to provideorientation and identity information. In an embodiment of the invention,the chest wall as imaged on a mammographic film is identified to provideorientation information.

As previously discussed, radiologists and other doctors becomeaccustomed to viewing radiographic films in a particular orientation. Byimplementing the method of the present invention and identifying thechest wall of a patient, a left or right orientation of a film can bedetermined so as to present films on viewing station 104 in a preferredorientation. For example, where a doctor is accustomed to viewing RCCfilms with the chest wall on the right-hand side, a digitized image canthen be rotated, if necessary to present the digitized image in apreferred left/right orientation. Those of skill in the art understandthat there exist many algorithms for rotating a digital image.

In another embodiment of the invention, the previously described leadmarkers 802, 804, 806 and 808 and the associated lead marker images 703,705, 707 and 709 are used to derive further orientation information. Amethod for identifying a lead marker image 703, 705, 707 or 709 on afilm is shown in FIGS. 14-16. The method involves optical characterrecognition (OCR) where the standard optical character recognitionproblem is composed of three stages: digitization, segmentation andrecognition. As digitization has already been discussed, FIGS. 14-16address segmentation and recognition.

As is evident from FIGS. 7 and 9, the image produced by the lead markeron the film is composed of distinct white characters on a darkbackground. In a preferred embodiment, these characters are not incontact with any other features in the image. Also in a preferredembodiment, the characters do not overlap the edges of the film or theedge of a label such as a case identification label or a patientidentification label. The segmentation process is composed of a numberof smaller tasks including finding starting points for locatingcharacters, thresholding, growing regions and extracting candidatecharacters.

FIG. 14 is a flowchart for a method for identifying candidate startingpoints. As noted in conjunction with FIG. 1, a digital mammogramtypically has a spatial resolution of 200 microns. This is usuallyconsidered to be a high resolution. For purposes of certain steps of anoptical character recognition algorithm, an image with a lowerresolution such as 400 microns or 1600 microns is adequate. Accordingly,at step 1402, a low resolution image is produced from the highresolution digitized image using methods known in the art. At lowresolution, all characters in the lead marker image tend to be segmentedas a single region. At step 1404, a gradient image is generated from thelow resolution image. A gradient image identifies changes in pixel grayvalues. Since lead marker images 703, 705, 707 and 709 are typicallybright images on a dark background, a high gradient exists in going froma dark background to the bright images and vice versa. In oneembodiment, a standard 3×3 Sobel gradient filter is used to create agradient image (i.e., changes in pixel gray values). The Sobel gradientfilter is described at page 501 in “Digital Image Processing” by WilliamPratt, Wiley-Interscience Publishers, 2nd Edition which is incorporatedby reference. Other methods that create a gradient image can also beused in the present invention.

At step 1406, a histogram-based low resolution gradient threshold imageis generated. This low resolution gradient threshold image containsthreshold values to be used as points of interest for the gradient imageregion growing of step 1408. The threshold values are determined byanalyzing histograms of local gradient values. The histograms arecomposed of structure indicating character edges (high gradient values)and flat image regions (low gradient values). The thresholds areselected such that these two regions are distinguished in the subsequentregion growing step 1408. High gradient features in the image which arenot characters will be removed using simple geometric constraints aswill be described for step 1410.

Using the points of interest, connected regions having high gradientsare identified at step 1408. Step 1408 is sometimes described as regiongrowing in the prior art. Pixels surrounding the points of interest arecompared to a predetermined gradient threshold. Pixels exceeding thepredetermined gradient threshold are then combined with the points ofinterest to form regions of interest. This process is then repeated foras long as there are pixels surrounding the regions of interest thatexceed the predetermined gradient threshold. As a result, each region ofinterest that is formed includes all contiguous pixels that exceed thepredetermined threshold.

At step 1410, geometric constraints are applied to the regionsidentified in step 1408. Lead markers are commercially available and aretherefore of known size. Thus, a geometric constraint can be imposed onthe size of the region of interest. In particular, the region ofinterest is compared to a minimum and maximum predetermined size; andwhere a region of interest is within the range between the minimum andmaximum size, the region of interest is identified at step 1412 as acandidate starting point. The candidate starting point identifies anoutline of a letter or set of letters such as used in lead markers. Thesteps as shown in FIG. 14 are repeated until all candidate startingpoints are identified.

Having identified candidate starting points that may correspond tooutlines of letters, the next task is to generate candidate characterregions by filling in potential outlines. FIG. 15 is a flowchart for amethod for identifying candidate character regions. At step 1502, a listof candidate starting points is input. At step 1504, a gray-valuethreshold is calculated based on the distribution of pixels of theoriginal gray-value image in the neighborhood of each starting point.For example, in the high resolution gray scale image, a region ofpredetermined size and shape (i.e. a square or rectangle) is created tosurround the candidate starting location. Then, in this region, thedistribution of gray-scale values is evaluated. In an embodiment, thedifference between a minimum pixel value and a maximum pixel value inthe pixels within the region of predetermined size and shape is comparedto a predetermined contrast value. This is essentially a minimumcontrast requirement for detecting images produced by lead markers forexample. Where the range of gray scale values does not exceed thepredetermined contrast value, a sufficient contrast does not exist andthe candidate starting location is removed from further consideration.Where the range of gray scale value exceeds the predetermined contrastvalue, a region growing process is performed at step 1506 in whichsimilar gray scale values surrounding the candidate starting locationare identified. Region growing constraints are applied at step 1508similar to those of step 1410. At this point, the candidate characterregions that may comprise the lead marker image have been filled in andare identified at step 1510.

FIG. 16 is a flowchart for a method for determining whether thecandidate character regions are letters from a lead marker image. Atstep 1602, the candidate regions are input. A binary image of eachcandidate region is then produced at steps 1604 and 1606. A value of −1is assigned to pixels outside the candidate region and within apredetermined region (i.e. a square or rectangle as in step 1504)surrounding the candidate character region at step 1604. A value of +1is assigned to pixels inside the candidate character region at step1606. Similarly, a binary image of a series of templates is used. Thetemplates are predetermined and can produced ahead of time. Thetemplates are letters or a sets of letters corresponding to lead markerimages. For example, four sets of letters can be “RCC,” “LCC,” “RMLO”and “LMLO” such that there nine distinct template letters including “R,”“R⁻¹”, “L,” “L⁻¹,” “C,” “M,” and “O” where the −1 superscript indicatesparity inversion (i.e., mirror image). As with the binary candidatecharacter region, the template is assigned a values of −1 outside thetemplate and +1 inside the template.

It is to be appreciated that when the candidate character region linesup perfectly with the template, pixels with values of +1 in thecandidate character region line up with pixels with values of +1 in thetemplate and pixels with values of −1 in the candidate character regionline up with pixels of −1 in the template. It is to be furtherappreciated that if the numerical values assigned to the lined up pixelsare multiplied together, all the products would be positive (i.e.,(+1)×(+1)=+1 and (−1)×(−1)=+1). Where there is misalignment or lack ofoverlap of the character region with the template, the numerical valuesassigned to some of the lined up pixels would have opposite sign andmany negative products could result (i.e., (−1)×(+1)=−1 or(+1)×(−1)=−1). Thus, the sum of all the products is a measure (or score)of how well the candidate character region lines up with the templatewith the highest score being achieved in case of perfect alignment andlower scores being achieved in other cases. Mathematically, this type ofmultiplication and addition is described as a dot product. For example,the binary values of the character region can be expressed as a vector Rand the binary values of the template can be expressed as a vector T andthe dot product of the vectors R and T yields the score, S.

Referring back to FIG. 16, at step 1608, a dot product is computed ofthe binary image of each candidate character region with a series oftemplates. Where the candidate character region lines up with thetemplate, the resultant dot product (or score) is maximized. It is to befurther appreciated that where the candidate character region and thetemplate do not line up, pixels with values of +1 and −1 will multiplythus reducing the dot product or score. At step 1610, the dot product iscompared to a predetermined score. Where the predetermined score isexceeded, the candidate letter is identified as the letter on thetemplate whose dot product is then being computed. At step 1614, a filmview is identified using the identified letters or groups of letters.For example, where the letters “R,” “C,” “C” or the set of letters “RCC”are identified, the film view is identified as the Right Cranial Caudalview. Similarly, the Left Cranial Caudal, Right Medio-Lateral Obliqueand Left Medio-Lateral Oblique film views are identified.

In the foregoing description of step 1608, the orientation of thecandidate letter and the template are assumed to be the same. This,however, is not always the situation. Thus, in a preferred embodiment,the orientation of each template is rotated through 360 degrees relativeto each candidate letter; and steps 1608-1612 are repeated at differentorientation angles to identify letters. As noted above, mirror-images ofthe template are also considered because the lead marker images may havebeen flipped by the technologist taking the image. Again, the mirrorimage template can be rotated through 360 degrees while performing thesteps of 1608-1812. In another embodiment, the orientation of theletters is determined by minimizing the moment of inertia of a candidateletter. The angle of the axis through the center of mass of a candidateletter which produces the minimum moment of inertia provides anorientation for the lead marker image. The template is then rotated tomatch this orientation and the steps of 1608-1612 are performed.

The flowchart of FIG. 17 shows a method for presenting films in apredetermined or preferred order and orientation. At step 1702, thedigitized images are input. The digitized images are separated by caseat step 1704 using, for example, the method of FIG. 11; laterality(i.e., left/right side) and orientation information is derived at step1706 using, for example, the method of FIG. 13; and, the specific filmview is identified at step 1708 using, for example, the methods of FIGS.14-16. At step 1710, the digitized images are then oriented in apreferred or predetermined orientation; and, at step 1712, the digitizedimages are sorted in a preferred or predetermined order. Thus, inoperating a CAD system of the present invention, the films may beinserted into the input mechanism such as film feeding mechanism 410 inany order and orientation; and the system will determine the orientationand order of the film images and display them on a display device suchas display panel 408 in a preferred or predetermined order andorientation.

The methods of FIGS. 14-16 have been described for identifying imagesproduced by lead markers, however, one of skill in the art understandsthat other images can be identified. For example, textual and othertechnical information is often included on the sides of the images or onpatient identification labels. The methods just described can also beused for identifying such information. Furthermore because patientidentification labels are placed on known regions within the films,patient identification labels can be used to identify the order andorientation of the films as well as to read in the information containedon the patient identification films.

As described above, visual cues and other information are used todetermine the orientation and identity of the various films produced inx-ray mammography. Visual cues are obtained by information inherent inan image such as the breast and chest wall characteristics in amammography film. Also, anatomical features derived from the films areused to determine orientation and identity. Information in an image caninclude the nipple or the skin outline. Furthermore, identifying markersplaced on the films are identified to provide orientation and identityinformation. Identifying markers can take the form of lead markerimages, bar code labels, patient identification labels or other types ofmarkers as may be developed. Still further, case separation is achievedby, among other methods, placing and identifying case separation cuessuch as special separator films between films of interest to provideinformation to the system or special markers on the first film of acase.

When the method of FIGS. 14-16 is applied to a large number of actualmammograms, errors occur when a film view cannot be identified or isidentified incorrectly. An error rate on the order of 5% is experienced.To reduce errors it is helpful to factor into the determination oforientation and location of labels other information that may beavailable such as:

-   -   the chest wall is either on the left side or right side of the        film;    -   the lead marker is in the top half of the film;    -   the lead marker image and the patient identification label        directions are defined relative to the center of the chest wall        (i.e., either clockwise or counter-clockwise);    -   the lead marker image is clockwise from the chest wall for right        films;    -   the lead marker image is counter clockwise from the chest wall        for left films;    -   the position of the patient identification label is always        opposite the chest wall;    -   the position of the patient identification label is always in        the top or bottom of the film; and    -   the patient identification label is clockwise from the chest        wall.

One of skill in the art understands that as specifications orassumptions are changed, certain key information can, nonetheless, beextracted from the changes. Redundancy occurs when two items ofinformation contain some of the same information. For example, theidentification of the film views of four films in a case provide thenecessary film view information for all the films. Identification ofthree film views in a case, however, also provides the necessary filmview information for all the films because the identity of the fourthfilm can be inferred. Thus, the identity of the fourth film providesredundant information.

Some information input to the CAD system is subject to human error andsome is not. In an embodiment of the invention, information not subjectto human error supercedes information that is subject to human error.For example, if a film side (L or R) as indicated by the lead marker isinconsistent with the label position, the side determined from the labelposition is preferred. In another embodiment, collective information ispreferred over single-film information. For example, if three filmsimply the identity of a fourth film and this result conflicts with thefourth film's lead marker, then the 3-film information is preferred.

Incomplete and thus erroneous information arises when one or more leadmarkers is not fully identified. For example, a lead marker may not befound on a film. This occurs for primarily two reasons: the radiationtechnologist placed the marker on or near a high gradient edge (i.e.,the patient identification label); or the exposure of the radiographicfilm is so long that the marker is “burned through” rendering littlecontrast on the film. Furthermore, it is possible for a technologist toplace the lead marker over a patient identification label to obscure thelead marker image. For example, for right side films, the patientidentification label is in the upper left hand corner and, therefore,competes for space with the lead marker image. In an embodiment of theinvention, this problem is alleviated by deriving information from thecase context. For example, where an RCC view is missing, but the RMLOview has been identified, the unidentified view can properly be labeledas RCC. Also, if the label position is determined, the orientation canbe derived. For example, where a patient identification label isidentified on the lower right hand side of a right side image, theorientation is determined.

Even if the direction of the patient identification label is notidentifiable, the digitized image can be processed as it was scanned andsubsequently processed after further information is obtained or derived.Further processing can include, for example, determination of the chestwall as described for FIG. 13. After such processing, the up/downorientation may remain ambiguous but further derived information, suchas the determination of a lead marker image, can make such adetermination possible. In such a situation, it may not be necessary tofully identify the lead marker image (i.e., whether it is “RCC” asopposed to “LCC”); basic information that a lead marker is in aparticular location regardless of its identity may suffice to determineorientation.

In gathering further error reduction and redundancy information, it hasbeen observed that two lead markers may not be identified for the tworight side views because of the proximity of the lead markers to thepatient identification label on the left side of the films. If twofully-identified films are found to be in their default order then itcan be assumed that missing films are also in their default order. Forexample, suppose the films are scanned in the order [x1, LCC, x2, LMLO]where x is unknown. In such circumstances, it is reasonable to assumethat x1 is RCC and that x2 is RMLO and process the case as such.

It has further been observed that a missing lead marker and a missingpatient identification label may occur. In such a situation, the missingfilm is processed in a clockwise orientation (i.e., the label isclockwise from the chest wall). In such a situation, however, theup/down orientation is ambiguous, but may be derived from furtherprocessing.

Conflicting information also creates errors. For example, conflictinginformation occurs when more than one film in a case is found with thesame lead marker. Such a situation occurs when there are extra films ina case or if two films have duplicate markers. Thus, there are twodistinct possibilities for duplicate lead markers: 3 identical sides or3 identical views. Assuming one film is incorrectly labeled, anincorrect film can be identified by the fact that its label directionwill be inconsistent with its lead marker side while the other film withthe same lead marker will be consistent with its label direction. If thelabel direction is not found for the films with duplicate markers, thenthe scan order is used to recover from this potential error. A correctidentity can be inferred based on the position provided the other threefilms in the correct scan order.

While preferred embodiments of the invention have been described, thedescriptions are merely illustrative and are not intended to limit thepresent invention. For example, although the embodiments of theinvention described above were in the context of a system forcomputer-aided diagnosis and detection of breast carcinoma in x-rayfilms, those skilled in the art will recognize that the disclosedmethods and structures are readily adaptable to broader application. Forexample, the invention is applicable to many other types of CAD systemsfor detecting other types of anatomical abnormalities, including but notlimited to chest x-ray, magnetic resonance imaging, and nuclearmedicine.

What is claimed is:
 1. An automated method for separating multipledigitized radiographic images, comprising the steps of: receiving in acomputer system multiple digitized radiographic images; identifying bythe computer system a plurality of case separation indicators embeddedin the multiple digitized radiographic images, each case separationindicator including a marker identifying an image as a first image of acase; and grouping in the computer system the multiple digitizedradiographic images into multiple cases using the plurality of caseseparation indicators, wherein the case separation indicator is receivedprior to a number of radiographic images in a case.